{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# StackingRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An ensemble-learning meta-regressor for stacking regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.regressor import StackingRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stacking regression is an ensemble learning technique to combine multiple regression models via a meta-regressor. The individual regression models are trained based on the complete training set; then, the meta-regressor is fitted based on the outputs -- meta-features -- of the individual regression models in the ensemble."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./StackingRegressor_files/stackingregression_overview.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Breiman, Leo. \"[Stacked regressions.](http://link.springer.com/article/10.1023/A:1018046112532#page-1)\" Machine learning 24.1 (1996): 49-64."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Simple Stacked Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.regressor import StackingRegressor\n",
    "from mlxtend.data import boston_housing_data\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.svm import SVR\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import warnings\n",
    "\n",
    "warnings.simplefilter('ignore')\n",
    "\n",
    "# Generating a sample dataset\n",
    "np.random.seed(1)\n",
    "X = np.sort(5 * np.random.rand(40, 1), axis=0)\n",
    "y = np.sin(X).ravel()\n",
    "y[::5] += 3 * (0.5 - np.random.rand(8))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error: 0.2039\n",
      "Variance Score: 0.7049\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD1CAYAAAC1BoUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XlYVOX///HnAAMqqKDFYLiiiCiEQZZaiaJoibvoV8tSf659UjOt3BH3FvVTqWVqqW1a7iWZuOOuH0xwIdwXEMatcUmBYZjfH16QCAgywwyceT+uyyvmzDlz3jdDrzlzn/vcR2U0Go0IIYRQJDtrFyCEEKLkSMgLIYSCScgLIYSCScgLIYSCScgLIYSCScgLIYSCOVi7gIfFxsZauwQhhCiTgoKC8l1eqkIeCi60KBISEvD19TVjNWWDrbYbpO222HZbbTcU3PbHHSBLd40QQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQihYqRtdI4QpdDodWq0WvV6PWq1Go9Hg6upq7bKEsBoJeaEYOp2O5ORksmfP1uv1JCcnA0jQC5sl3TVCMbRaLY/eHsFoNKLVaq1UkRDWZ3LIp6Sk8Oabb9K+fXvCwsJYvnx5nnWMRiPTp08nNDSUjh07cuLECVN3K0Qeer3+iZYLYQtM7q6xt7dn7NixNGrUiLt379K9e3deeukl6tWrl7NOTEwMFy5cIDo6mri4OCIjI1m1apWpuxYiF7VanW+gq9VqK1QjROlg8pG8u7s7jRo1AsDFxQUvL688X4+3bdtGly5dUKlUNG7cmNu3b3P16lVTdy1ELhqNBpVKlWuZSqVCo9FYqSIhrM+sJ16TkpJISEggICAg13KtVouHh0fOYw8PD7RaLe7u7nleIyEhodj7T0tLM2n7sspW2w1Fa3tKSgopKSkWqshybPV9t9V2Q/HabraQ/+effxgxYgTjx4/HxcWl2K9jysRDtjpxka22G6Tttth2W203WHGCMr1ez4gRI+jYsSNt27bN87xGoyE1NTXncWpqqnyFFkIICzA55I1GIxMmTMDLy4v+/fvnu05ISAjr16/HaDRy9OhRKlasmG9XjRBCCPMyubsmNjaWDRs2UL9+fTp37gzAqFGjuHLlCgC9e/cmODiYXbt2ERoaSvny5Zk5c6apuxVCCFEEJof8888/T2Ji4mPXUalUTJ482dRdCSGEeEJyxasQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYWUJ+3LhxNGvWjA4dOuT7/MGDBwkKCqJz58507tyZ+fPnm2O3QgghCmHy7f8AunXrRp8+fRgzZkyB6zz//PN8/fXX5tidEEKIIjLLkXyTJk2oXLmyOV5KCCGEGZnlSL4ojh49SqdOnXB3d2fMmDF4e3tbateKodPp0Gq16PV61Go1Go0GV1dXa5clhCjFVEaj0WiOF0pKSmLo0KFs3Lgxz3N3795FpVLh7OzMrl27mDFjBtHR0XnWi42NpUKFCsWuIS0tjXLlyhV7+9IsKyuL/N4qlUpFRkaGYttdGCW/54Wx1bbbaruh4Lbfu3ePoKCgfLexyJG8i4tLzs/BwcFMmTKFmzdvUqVKlTzr+vr6Fns/CQkJJm1fmiUmJqLX6/Msd3BwwM7OTrHtLoyS3/PC2GrbbbXdUHDbY2NjC9zGIkMor127lnMUGh8fT1ZWFm5ubpbYtWLkF/CPWy6EEGCmI/lRo0Zx6NAh/v77b1q0aMHw4cPJzMwEoHfv3mzevJkVK1Zgb29PuXLlmDt3LiqVyhy7thlqtTrfQFer1WRlZVmhIiFEWWCWkJ87d+5jn+/Tpw99+vQxx65slkajITk5OVe/vEqlQqPRkJKSYsXKhBClmVzxWka4urri6emJWq0GHhzBe3p6yugaIcRjWWwIpTCdq6urhLoQ4onIkbwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYhLwQQiiYWUJ+3LhxNGvWjA4dOuT7vNFoZPr06YSGhtKxY0dOnDhhjt0KIYQohFluGtKtWzf69OnDmDFj8n0+JiaGCxcuEB0dTVxcHJGRkaxatcocuxZWptPp0Gq16PV67OzsUKlUGAwG1Go1Go1GbnIihJWZJeSbNGlCUlJSgc9v27aNLl26oFKpaNy4Mbdv3+bq1au4u7ubY/fCSnQ6Xa77zj58Q3G9Xk9ycjJAnqB/+INBPgyEKFkW6ZPXarV4eHjkPPbw8ECr1Vpi16IEabXaXDcWf5TRaMzzPmd/MOj1euDfDwOdTleitQphq0rdPV4TEhKKvW1aWppJ25dV1mq3wWAodB29Xp+rtvy2MRqNJCUlkZKS8sQ12Op7DrbbdlttNxSv7RYJeY1GQ2pqas7j1NRUNBpNvuv6+voWez8JCQkmbV9WWavdiYmJOUfkBVGr1fj4+OQ8Pn78eIHrFrUNj3b3VKtWzSa7e+Tv3fYU1PbY2NgCt7FId01ISAjr16/HaDRy9OhRKlasKP3xCqDRaFCpVAU+r1Kp8nyYq9XqfNctaPmjHu3uAaS7R4jHMMuR/KhRozh06BB///03LVq0YPjw4WRmZgLQu3dvgoOD2bVrF6GhoZQvX56ZM2eaY7fCyrKPnp9kdI1Go8l1shby/zAoSH7nAbL7/m3xaF6Iwpgl5OfOnfvY51UqFZMnTzbHrkQp4+rq+kTh+ugHw5OOrimoe0iv16PT6bhy5UquUT729vY2250jBJTCE6/iyWVlZeX0j1tySOLNf27y56U/OXHlBP+k/0N6ZjrpmekYsgyUV5fHpZwLLk4P/Xv4cRUXqleqjrOT8xPtU61W5xv0dnZ2+Q7jNRgMBQ7lFMIWSMiXYdlHrkajMc+QRDBvqF2/c50jl44QezGW2IuxHLl0hPPXz5v8unWfrsuz1Z/N+RdQPYA6T9XBzi7/00UFdfc87tyAdOcIWyYhX0Y9eiHSw0wNNe1tLUcuHskJ89iLsVy6eSnPeuUdyxNQPYCA6gG4VnDFycEJJwcn7O3sSdOncTf97r//0u7menwn7Q5XdFc4e+0sZ6+dZd2f63Je19nJGX9P/3/D3/PBfytXqJynuwfA09PzsRfjQcHdPEIonYR8GVXYhUhFDbUUXUquo/PYi7Ek65LzrFfBsQLP1XyOoFpBBNYMJKhWEA08GuBgX/w/oYzMDBJTE4lPin/wL/nBf6/ornDg3AEOnDuQa/1aVWvlCv0mtZuQdj0NV1fXXKGfn6KO3hFCaSTky6iijE9/mNFoJPnvZGIvxeYcpcdeiiX1VmqebSuWq/gg0GsGEVjrQaDX19TH3s7erG1wdHDEv7o//tX9eYM3cpZfv3M9V+jHJ8VzPPk4F29c5OKNi/wW91vOul5VvOj6fFdaerXE08ETtX3eMH+S0TtCKI2EfClV2PwuBZ2AzJbumM7aI2sfBPqlB0fq1+5cy7Ne5fKVCawZ+CDMawYRVCuIeu71CuwTt4SnKj5FiG8IIb4hOcsyDZmcuXqG+KR44pLiiLscx54zezh38xxzoucwhzlUKleJZjWa0aJOC4LrBFO5XGUZXSNsnoR8KfRof3t+J1MfPgGZnpnOvkv7iEuJI+FaAn9d+4ub927meV23Cm65wjywViBeT3lZNdCLysHegQbVGtCgWgN6NukJgD5Tz4rtKzh26xhRx6JISElg8+nNbD69GQd7B8IDw5kQNkECXtg0CflSqCgX/FSqVIk9Z/ewdM9Sok9Fczfjbq71q7pUzRXmQTWDqP1U7ceOQilr1A5qmtRowltt3+LTHp9y7to5ouKj+DXuV3Yk7mDl4ZWsPLySboHdmBg2kedqPmftkoWwOAn5UuhxF/ycvXqWZfuW8f2B77l442LOc8/VfI72fu0JqvUg2GtUqaGoQC8Kr6e9GN56OMNbD+fyzct8/MfHLNm9hLVH1rL2yFo6PNuBiWETedHrRWuXKoTFSMhbUUH97o/2t+sNenad38Wq46vYd3FfzvKaVWryZtM3aebejLCXwqzRhFKrRpUazH99PhPaT2B29GwW7lrIxviNbIzfSGjDUCaFTeKV+q9Yu0whSpyEvJU8rt89u7895XYKq46vYt2JdVz758FJ03LqcvR8vif9mvcjuH4wdnZ2NjvtalFUc63GnJ5zGPvaWP675b/M3zGfLSe3sOXkFlrUb8GksEm09m1tc996hO2QkLcCnU6X78U72f3ud5zuMGvnLDbEbcBgfDD/uo+7D2+3epu3mr2Fm7ObpUsu856u+DQzu83k/Xbv88W2L/h82+fEnIoh9FQoTb2aMqnDJF7ze03CXiiOhLyFZR/BP8qQZWDX+V0sP7KcI1eOAGBvZ0+v53vxdsu3ecX7FQkgM6jiXIXITpG81+Y9FuxYwNytczlw7gBhX4QRVCuIiWET6RTQqUyMOBKiKCTkLezRkTP39ffZcHID3//5PZduPZg6oFL5SgxpMYThIcOpUaWGtUpVtMoVKjM+bDwjWo/g65iv+XTzp8RejKXrl13x9/RnQtgEwoPCzX4BmBCWJiFvYdknVG+l3WJF3Ap+PPojurQHN7x4puIzDGs1jGGhw6hYrqI1y7QZLuVcGN12NP9p+R+W7F7Cx398zLHkY/Ra1IsGHg0Y3348vV/obdL0DUJYk3wntbD0rHS+OvAV7Za2Y8GBBejSdPhr/Pn0tU+JmxDHuI7jJOCtoLxjeYa3Hs7ZmWdZ2GchtavW5q/Uv3jr27fwmeTDkt1LyMjMsHaZQjwxCXkLuZd+j083f0rbb9ry5cEv+SfjH5rWaMq33b/lp14/MTBkIE9VfcraZdo8J7UTQ4KHcGr6KZb2W4q3uzfnrp1j0HeD8J7gzZc7viRNn2btMoUoMgn5EpaRmcGCHQuoN6EeH67+kJv3btK0TlN+6PUDi7stpnmd5lSvXl0uvS9l1A5q+r3Uj5NTT/LjwB9pWK0hl25e4p2f3sFrnBf/3fJf7qXfs3aZQhTKLCEfExNDu3btCA0NZdGiRXmeX7t2LU2bNqVz58507tyZVatWmWO3pVqmIZNle5fhM9GHYT8NI+VWCkG1gtj07ib2jdvHG63fwM/PDx8fHwn4UszB3oHXX3ydY5HHWDV0FQHVA0i5lcKoX0ZRe1xtPt70MXfS7li7TCEKZPLZJIPBwNSpU1m6dCkajYbw8HBCQkKoV69ervXat29PRESEqbsr9bKyslhzZA0RGyL4K/UvAHyr+TK9y3S6PtdVhkGWUXZ2doQHhdM9sDsb4zcybeM0Dl84zNi1Y/lk8yeMbDOS4SHDca0gH9iidDE55OPj46lVqxY1ajwY6hcWFsa2bdvyhLxSZU9NkJGRwb7L+1hwcAHHrhwDHsylEtkxktdffF2G4imESqWiY0BHOjzbgegT0UyLmsbeM3uJ2BDB7OjZDG81nPdC36OqS1VrlyoEYIaQ12q1eHh45DzWaDTEx8fnWS86OprDhw9Tp04dxo0bR7Vq1fJ9PVMu0U9LS7PoJf5ZWVkYjUYOJx3mi31fcDTlKADuzu4MbTqUbn7dcLR35FTiqRKtw9LtLk2s2faa9jVZ1HERhy4fYuHBhRy8fJAZv89g7pa59A7oTb+gflQpXyXP/WjNdaGVrb7vttpuKF7bLTL4t1WrVnTo0AFHR0dWrlzJmDFj+O677/Jd19fXt9j7SUhIMGn7JxW1N4pPdn5CzIUYANzKuzHg+QH0CexDgF+AxeqwdLtLk9LQ9oYNG9KvXT/2ntnL9Kjp/HH8D77937esiFtB90bd6RfUD43LgztTZQe+OW5kUhrabg222m4ouO2xsbEFbmNyyGs0GlJT/72FnFarzXOrNTe3f+da6dGjB59++qmpu7WK7K4Z7S0tXx36il/ifyHLmIWzozN9A/vy1nNv4ezobO0yhZW8VO8lNr27icPnDzPj9xlsOLqBH47+wM/HfqbXs714p+k7ODs6YzAY8twERoiSYvL3Rn9/fy5cuMDly5fJyMggKiqKkJCQXOtcvXo15+ft27dTt25dU3drcTqdjguXLrDk4BLCloexMm4lAD39exLVN4q3X3w7J+DlptG2rUmdJqx/Zz2rX19NW++2ZBoy+f7P7+n0fSe2nd0G/DsZnRAlzeQjeQcHByIiIhg4cCAGg4Hu3bvj7e3N559/jp+fH61bt+b7779n+/bt2NvbU7lyZWbNmmWO2i1q3aF1TN82nXM3zwHwcq2XGf3KaOpVzX2CWW4aLbL5PePHnPZzSLiawNTtUzmuPc7IjSMJrRfKuJbjeNr5aWuXKGyAWfrkg4ODCQ4OzrXs3Xffzfl59OjRjB492hy7sriLNy4y+pfRrDmyBoCalWsyJngMLeq0yFkn+yYf+d1wW9iu7PsC+Lr78kPPH1gZv5LP933OljNbOHD5AGNajqFRo0YyrFaUKJl1qQD3M+7z6eZP+eiPj7ifcZ/yDuUZ/MJg3nruLRwdHHPWU6vV+Pj4WLFSUVplf9hfuXIFgDcav0GIVwhTt09lz8U9TNw8kc1nNjOx5USecXlGDhJEiZCQf4TRaOTXuF8ZuXIkF25cAKBXk15MbDsR/iHPcDjpmhGP4+rqiqura85J+2qVqrE4fDF7UvYwdsNYdp/dTacLnRjefDhvPvemnJAVZich/5DE1ETeXfkum09sBsDf0595vecR7POgK6qge7IKUZjssM/WoEED/Fz9mLl9Jr/99Ruzd89m9/ndTG87HQcHB/m7EmYjIQ/cSbvDtI3T+GzrZ+gNelwruDK101Tebvl2rnnEH/0fVQhTuKhdmNluJu282xGxNYKDSQfp/mN3JreeLF2AwmxsehZKo9HIjwd+xGeiD59u/pTMrEwGvjKQU9NPMbz1cLlRhChR2UNtg72CWfPGGl6p/Qq3028z+vfR9F/aXyY+E2ZhsyF/9NJRWnzSgj7f9CHlVgov1HmBg+MOsvitxTxdUYa2iZKn0WhyRtY85fwUCzotYGKriZRzKMeyfctoPLUx+8/ut3KVoqyzuZC/ff82I1eOJGh6EHvO7MG9ojvf9vuW/WP306ROE2uXJ2yIq6srnp6eOUf0jo6OjG4/mthJsTSu0Zhz187xyievEPlrJJmGTCtXK8oqm+mPMBqN/PK/X3jv5/dIuZWCncqOEa1HMKXTFJkeVlhNfud5XF1dOTDuAJM2TGJ29Gym/DaF6JPR/DDgB7ye9rJSpaKssokj+dPa07T7rB29FvUi5VYKTWo2Yc2baxjkNwjtZS06nc7aJQqRi5PaiU/CP2HbqG14unqy/+x+AqYEsGzvslzDeIUojKJDPk2fRuSvkfhH+rPl5BbcKrjxWfhnfNP1G+q5PZiOQK/Xk5ycLEEvSqVWDVoRHxlPj6Ae3E2/S/9l/en5dU9u/nPT2qWJMkKxIb/l5Bb8I/2Z8tsU0jPT6de8H4nTE3m19quoyH0ZuUwWJUqzKs5V+HnIzyzrvwwXJxdWx67m2chnOXDpgLVLE2WA4vrkU2+lMuqXUaw4tAKAhtUa8kmXT6jnXA/txYKDXK/XW6pERZMLxkqGSqWib/O+vFzvZd789k32n93PgNUDOHn7JNO7TMdJ7WTtEkUppZgjeUOWgRVHV9BgUgNWHFpBecfyzOo2ix0jdlDbqXahIS7TA5tOp9ORnJyc87uWrjDzq+tel5gPYojsGImdyo7Z0bNpOqspCSm2eackUThFhLw+U0+buW2Ytn0at+7for1/e05EnmBos6FcTb1a6IkqmYPGPLRabZ7ftXSFmZ+DvQOTO03mh/97MNrm6OWjBE4LZMGOBXJSVuShjJA36Dl6+SgaFw1r3l7DxuEbcVO75Uz29DhqtRpPT0/pUjCDgr4tSVdYyQh4JoCjEUfp/1J/0vRpDPtpGB3mdUB7Wz5Uxb8UEfIVnCpw8aOLRA+IpltgN1QqVb5HlQ9Tq9X4+fnh4+MjAW8mBXV5SVdYyalYriLf9vuWVUNX4VbBjd+P/Y5/pD8b4zZauzRRSigi5AEqla+E2v7fMHnc0aN0z5SMhy/Tzya/a8sIDwonfnI8IQ1CuHbnGh3nd+Q/P/6He+n3rF2asDKzhHxMTAzt2rUjNDSURYsW5Xk+IyODkSNHEhoaSo8ePUhKSjLHbh/rcUeP0j1TMh69TF+6wiyrepXqbHlvC7N7zMbRwZGvdn5F0PQgjlw8Yu3ShBWZHPIGg4GpU6eyZMkSoqKi2LhxI2fOnMm1zqpVq6hUqRJbtmyhX79+zJ4929TdFqqgo8rq1atL6JQgV1dXfHx8pCvMSuzs7BjddjQHxx3Et5ovf6X+RdNZTfnkj08wZBmsXZ6wApNDPj4+nlq1alGjRg0cHR0JCwtj27ZtudbZvn07Xbt2BaBdu3bs37+/xEcByFGlsGWNazYmdmIsw1oNQ2/QM2bNGNrMbcPlm5etXZqwMJNDXqvV4uHhkfNYo9HkGTKn1WqpVq0aAA4ODlSsWJG///7b1F0XSo4qhS0r71ieea/P4/cRv6OppGFn4k6enfIsPx/+2dqlCQsqdVe8JiQU/6KOtLQ0k7Yvq2y13SBtL0rbazvUZvXrq4mIjmDHuR30WtSLn3b/xMSQibg4ueSsl5WVlecexnZ2pW9shrznT9Z2k0Neo9GQmpqa81ir1eYZTaHRaEhJScHDw4PMzEzu3LmDm5tbvq/n6+tb7FoSEhJM2r6sstV2g7T9Sdq+LXAbX+/6mlGrRvFrwq8cu3aMHwb8QPN6zXOuVn5UtWrVSt03YHnP87Y9Nja2wG1M/pj29/fnwoULXL58mYyMDKKioggJCcm1TkhICOvWrQNg8+bNNG3aNM9JUSFEyVKpVAxtOZQjE48QWDOQ89fP88onrxCxIYKkK0lytbJCmRzyDg4OREREMHDgQNq3b89rr72Gt7c3n3/+ec4J2PDwcHQ6HaGhoSxdupT333/f5MKFEMXToFoD9o/bz5hXx2DEyLSN03hjxRtc0l3Ks65crVz2maVPPjg4mODg4FzL3n333ZyfnZyc+OKLL8yxKyGEGTg6OPJR94941e9V3vr2LeJT4wn/KZxxwePo0rBLzjdtuVq57Ct9Z1WEEBbT0qclcRFxdA3oyn39fSK2RjD699HcSrslVysrhIS8EDbOzdmNNe+sYWGvhTg7OrPlzBZ6/NSDSxmXSt1JV/HkJOSFEKhUKoa0HkJ8ZDwv1nmRlDspdPiqA1N+nUKmIdPa5QkTSMgLIXJ4Pe3F7g93M+61cRgxEvlbJK1mt+LSjbwnZUXZICEvhMhF7aBmZreZbHlvC9UqV2PPmT0ETA1gdexqa5cmikFCXgiRr9a+rYmbHEeHZzugu6ejx8IeDFw+kLtpd61dmngCEvJCiAI9XfFpfh32K/N6z8PJwYlv9nxD4LRADp8/bO3SRBFJyBeTTqcjMTGR48ePk5iYKDerFoqj0+k4efIkJ06coKV7S355/RcaejTk9NXTNP+4ObN+nyXTF5cBEvLFkD3PR/bVgHq9nuTkZAl6oRg6nY6kpCSysrJylnm5efFd9+/o07gPmYZMxq8bT8icEDkpW8pJyBdDfvePlXk+hJIU9Lfs5ODEmOAxLOyyEHcXd2JOxShq+mIlfkOXkC+GgubzkHk+hFIU9rf8Uq2XWNV7FZ0COnHr/i16LepF32/7cvv+bQtVaH5K/YYuIV8MBc3nIfN8CKUoyt9ylQpVWP/Oehb2WUh5x/J8t/87Gk9tzN4zey1Qofkp9Ru6hHwxFHT/WJnnQyhFUf+WVSoVQ4KHcGTiEZ6r+Rznr5+nxSctGLtmLOn69BKu0ryU+g1dQr4Y5P6xQulcXV2pXr36Y+8M9fCNfxpUa8CBcQcY99o4AD7+42OazGjC0UtHS7xWc1HqN3QJ+WKS+8cKpXN1daVhw4b4+fnluZObm5sbnp6euZY5Ojgys9tM9ozZQz33ehxLPsYLM19g1u+zysT8N0r9hi4hL4QolKenJ35+fjn/Hg34hzWr24yjEUf5T8v/oDfoGb9uPK988gqntactWPGTU+o3dAl5IYTZOTs5s+CNBWweuRlPV08OnDtAwNQAFuxYkGvsfWmjxG/oJoW8Tqejf//+tG3blv79+3Pr1q181/P19aVz58507tyZoUOHmrJLIUQZ0rZRW45FHqNP0z7cz7jPsJ+G0e6zdly+ednapdkMk0J+0aJFNGvWjOjoaJo1a8aiRYvyXa9cuXJs2LCBDRs2sHDhQlN2KYQoY9yc3fh+wPesHrqaqi5V2ZqwlYYRDZm/fb5Mi2ABJoX8tm3b6NKlCwBdunRh69atZilKCKE83YO6czzyOF2f68rd9LsMXzGclz56iWNJx6xdmqKZFPI3btzA3d0dgKeffpobN27ku156ejrdunWjZ8+e8kEghA3zqOzB2v+sZe3ba3nG9RkOnj9I4PRAxq8dz/2M+9YuT5FUxkcv8XpEv379uH79ep7lI0eOZOzYsfzvf//LWdakSRMOH847BalWq0Wj0XD58mX69u3LsmXLqFmzZp71YmNjqVChQnHaAUBaWhrlypUr9vZlla22G6TtZbntd9Lv8Nmez1gZtxIjRmpUrkFkaCTNajZ77HZlvd2mKKjt9+7dIygoKN9tHAp70WXLlhX4XNWqVbl69Sru7u5cvXqVKlWq5Lte9jjTGjVq8MILL3Dy5Ml8Qx4enKQtroSEBJO2L6tstd0gbS9rbdfpdGi1WvR6PZXLV+bLPl8y7NVhDP5+MCeunGDA6gH0bdaXOT3nUNWlar6vURbbbS4FtT02NrbAbUzqrgkJCWH9+vUArF+/ntatW+dZ59atW2RkZABw8+ZNjhw5Qr169UzZrRCiDCpoArCGTzXkyKQjTO8yHScHJ5bvX06DSQ344cAPeeaSEU/OpJAfPHgwe/fupW3btuzbt4/BgwcDcOzYMSZMmADA2bNn6d69O506daJv374MGjRIQl4IG/S4CcAcHRyZEDaB+MnxtPRpyfVJ+x+KAAANkklEQVS713nzmzd59bNXOXftnJUqVoZCu2sex83NjeXLl+dZ7u/vj7+/PwCBgYH89ttvpuxGCKEARZkArL5HfbaP3s6yfcsY/ctook9G4xfpx5ROU3ivzXs42JsUWTZJrngVQlhEUScAU6lU9H+pP39N+4vXX3id+xn3+XD1hzSZ0UTuLVsMEvJCCIt40gnA3Cu58+OgH9n07iZqV63N0ctHaTqrKR/t/Ig7aXcsUbIiSMgLISyiuBOAver3KsenHGd029EAfHfkO7wneLNk9xK5YrYIJOSFEBZT3AnAnJ2cmd1jNocnHOZZj2fR3tYy6LtBBE4LZOtJucDycSTkhRBlRmCtQFb0XsGKQSuoWaUm8UnxhP43lA5fdCAhJcHa5ZVKEvJCiDJFpVLR64Ve/DXtL2Z1m0XFchWJOhaFf6Q/w34axrU716xdYqkiIS+EKDN0Oh0Gg4Hjx49z6fwlhjYbyukZpxnSYghGo5EFOxbgPcGb2Ztnl7l7zJYUCXkhRJmQfcVstuwrZp2ynFj45kLiJsfRrlE7bt2/xQerP8A3wpdfDv9Sqm9SYgkS8kKIMuFxV8wC+Hn68cfIP9j07iYaVmvI+evn+b9F/0fA1ABWHlppsyNxJOSFEGVCUa6YhQdDLuMmx7Gwz0I8XT05nnyc3ot70zCiIcv3LUefmf/rKJWEvBCiTCjqFbMADvYODAkewtmZZ/n6za+pXbU2p7Sn6Le0Hz6TfFgUs8hm+uwl5IUQZcKTXDGr0+lITEzkdOJpgjXBHPrgEMv6L6O+pj7nr59nyPdDqDehHvO2zVP8zUok5IUQZUL2FbPZCrpiNr8pja+mXqVzw86cnHqSFYNW0OiZRiT9ncSIlSOoM64OszfP5m7aXYu2x1Ik5IUQZYarqyv29vaPvWL2cSdo7e3s6fVCL+Inx7P27bUE1gxEe1vLB6s/oPa42syImsGte7cs1RyLkJAXQihKUU7Q2tnZ0TWwK/+b+D+iRkTR1KspN+7eYOL6idQaW4v3V73P2atnLVVyiZKQF0IoypOcoFWpVLT3b8++sfvYOmorwfWDuXX/FnOi5+A90ZuwL8LYdGxTmR5rLyEvhFCUJ53SOPv5oGpBfBn2JSv+bwWdfDuhtlPz+7Hfaf9Fe+pPrM/Hmz4uk3epkpAXQihKcaY01ul0JCUlkZWVhZ+HHzPazmDrgK2899J7eFb25Oy1s4xdO5a64+vSOLIxs36fxWntaUs1ySQm3Utr06ZNzJ8/n7Nnz7Jq1aqcW/49KiYmhhkzZpCVlUWPHj1y7gUrhBAlwdXVtcjTGAM5V80+zK28G//v+f9H38C+7L6wm98Tf2fX+V3EJccRty6O8evGE1A9gPCgcMKDwmlQrUGR9qXT6dBqtej1euzs7FCpVBgMBtRqNRqN5onqLgqTQr5+/frMmzePyZMnF7iOwWBg6tSpLF26FI1GQ3h4OCEhIXIzbyFEqVHQyVoAezt7Wnq1pKVXS9Iz09l3aR9bz25l5/mdxCXFEZcUx6QNk2j0TKOcwG/0TKM8XUbw7/DO7NE/D/f1Z8/FA5g16E0K+bp16xa6Tnx8PLVq1aJGjRoAhIWFsW3bNgl5IUSpoVarHxv02ZwcnGjl1YpWXq3w9vFma8JWVseuZv3R9Zy4coITV04w5bcpNPBokBP4z1Z/Nifw8xve+bDsoZ6lJuSLQqvV4uHhkfNYo9EQHx9f4PoJCcWf+D8tLc2k7csqW203SNttse0l0e7ijJ45d+YcXmovPmz6ISObjOTQ5UNEn4pm65mt/JX6F9OjpjM9ajo1XWvS1rst7eq3w6eqT75H+A/T6/UFtq84bS805Pv168f169fzLB85ciRt2rR5op0Vha+vb7G3TUhIMGn7sspW2w3Sdltse0m1W6fTceXKlZzAt7e3p1KlSuh0ulxH3yqVKt8TuQF+AQx6bRCZhkx2Ju5kdexq1v65lku6Syw5vIQlh5fgWcmT0HqhhHqH4q/xzzfw1Wo1Pj4++dZYUNtjY2MLbFehIb9s2bLCVnksjUZDampqzmOtVvvYoUxCCGENBZ2sdXZ2zjlRWpSTow72DrRp2IY2Dduw4I0F7D69m9Wxq1lzZA3Jt5JZdmQZy44so0r5KgR5BhHkGUQrr1Y8U+mZQod6FkeJd9f4+/tz4cIFLl++jEajISoqijlz5pT0boUQwiyedKTOw+zt7Gnp05KWPi35vNfn7D+7nx/2/cCGuA2k3klly5ktbDmzhXn753Fw2EE8qz1+qGdxmBTyW7ZsYdq0ady8eZMhQ4bg6+vLN998g1arZeLEiSxevBgHBwciIiIYOHAgBoOB7t274+3tba76hRCiTLC3s+dl75d52ftlvjJ+xZmrZ4g5FUPM6RiqVa5GwwYNC+2vLw6TQj40NJTQ0NA8yzUaDYsXL855HBwcTHBwsCm7EkIIxVCpVHhrvPHWeDPglQElui+54lUIIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMQl4IIRRMESGv0+lITEzEYDCQmJiITqezdklCCFEqmHRnqE2bNjF//nzOnj3LqlWr8Pf3z3e9kJAQnJ2dsbOzw97enrVr15qy21x0Oh3Jyck5d1PX6/UkJycDmP1eiUIIUdaYFPL169dn3rx5TJ48udB1ly9fTpUqVUzZXb60Wm1OwGczGo1otVoJeSGEzTMp5OvWrWuuOopNr9c/0XIhhLAlFuuTHzBgAN26dePnn3826+uq1eonWi6EELak0CP5fv36cf369TzLR44cSZs2bYq0kxUrVqDRaLhx4wb9+/fHy8uLJk2a5LtuQkJCkV4zW1ZWVr7LMzMzn/i1yqq0tDSbaeujpO2213ZbbTcUr+2FhvyyZcuKW08OjUYDQNWqVQkNDSU+Pr7AkPf19X3i19fpdGi1WvR6PWq1Go1GY1P98QkJCcX6vSmBtN322m6r7YaC2x4bG1vgNiXeXXPv3j3u3r2b8/PevXvx9vY26z5cXV3x8fHB3t4eHx8fmwp4IYR4HJNCfsuWLbRo0YI///yTIUOGMGDAAODBiJdBgwYBcOPGDV5//XU6depEjx49CA4OpkWLFqZXLoQQolAmja4JDQ0lNDQ0z3KNRsPixYsBqFGjBr/++qspuxFCCFFMirjiVQghRP5MOpIXQgiQwQ+lmYS8EMIkMrVI6SbdNUIIkzxuahFhfRLyQgiTyNQipZuEvBDCJDK1SOkmIS+EMIlGo0GlUuVaplKpcq50F9YlJ16FECbJPrkqo2tKJwl5IYTJXF1dJdRLKemuEUIIBZOQF0IIBZOQF0IIBZOQF0IIBZOQF0IIBVMZH70e2Yoed3cTIYQQBQsKCsp3eakKeSGEEOYl3TVCCKFgEvJCCKFgirjiNSYmhhkzZpCVlUWPHj0YPHiwtUuyiHHjxrFz506qVq3Kxo0brV2ORaWkpPDhhx9y48YNVCoVPXv2pG/fvtYuq8Slp6fzxhtvkJGRgcFgoF27dowYMcLaZVmUwWCge/fuaDQavv76a2uXYzEhISE4OztjZ2eHvb09a9euLdqGxjIuMzPT2Lp1a+OlS5eM6enpxo4dOxpPnz5t7bIs4tChQ8bjx48bw8LCrF2KxWm1WuPx48eNRqPReOfOHWPbtm1t4n3Pysoy3r1712g0Go0ZGRnG8PBw459//mnlqizr22+/NY4aNco4ePBga5diUa1atTLeuHHjibcr89018fHx1KpVixo1auDo6EhYWBjbtm2zdlkW0aRJEypXrmztMqzC3d2dRo0aAeDi4oKXl5dN3KRCpVLh7OwMQGZmJpmZmXlmgFSy1NRUdu7cSXh4uLVLKTPKfMhrtVo8PDxyHms0Gpv4n138KykpiYSEBAICAqxdikUYDAY6d+5M8+bNad68uc20G2DmzJl88MEH2NmV+egqlgEDBtCtWzd+/vnnIm9jm78poRj//PMPI0aMYPz48bi4uFi7HIuwt7dnw4YN7Nq1i/j4eE6dOmXtkixix44dVKlSBT8/P2uXYhUrVqxg3bp1LF68mB9//JHDhw8XabsyH/IajYbU1NScx1qtVm5WYCP0ej0jRoygY8eOtG3b1trlWFylSpV48cUX2b17t7VLsYgjR46wfft2QkJCGDVqFAcOHOD999+3dlkWk51rVatWJTQ0lPj4+CJtV+ZD3t/fnwsXLnD58mUyMjKIiooiJCTE2mWJEmY0GpkwYQJeXl7079/f2uVYzM2bN7l9+zYAaWlp7Nu3Dy8vLytXZRmjR48mJiaG7du3M3fuXJo2bcrs2bOtXZZF3Lt3j7t37+b8vHfvXry9vYu0bZkfQung4EBERAQDBw7MGVpV1MaXdaNGjeLQoUP8/ffftGjRguHDh9OjRw9rl2URsbGxbNiwgfr169O5c2fgwe8jODjYypWVrKtXrzJ27FgMBgNGo5FXX32VVq1aWbssUcJu3LjBO++8Azw4J9OhQwdatGhRpG1lWgMhhFCwMt9dI4QQomAS8kIIoWAS8kIIoWAS8kIIoWAS8kIIoWAS8kIIoWAS8kIIoWAS8kIIoWD/H5jIkcJ6FPEHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Initializing models\n",
    "\n",
    "lr = LinearRegression()\n",
    "svr_lin = SVR(kernel='linear')\n",
    "ridge = Ridge(random_state=1)\n",
    "svr_rbf = SVR(kernel='rbf')\n",
    "\n",
    "stregr = StackingRegressor(regressors=[svr_lin, lr, ridge], \n",
    "                           meta_regressor=svr_rbf)\n",
    "\n",
    "# Training the stacking classifier\n",
    "\n",
    "stregr.fit(X, y)\n",
    "stregr.predict(X)\n",
    "\n",
    "# Evaluate and visualize the fit\n",
    "\n",
    "print(\"Mean Squared Error: %.4f\"\n",
    "      % np.mean((stregr.predict(X) - y) ** 2))\n",
    "print('Variance Score: %.4f' % stregr.score(X, y))\n",
    "\n",
    "with plt.style.context(('seaborn-whitegrid')):\n",
    "    plt.scatter(X, y, c='lightgray')\n",
    "    plt.plot(X, stregr.predict(X), c='darkgreen', lw=2)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "StackingRegressor(meta_regressor=SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1,\n",
       "  gamma='auto_deprecated', kernel='rbf', max_iter=-1, shrinking=True,\n",
       "  tol=0.001, verbose=False),\n",
       "         refit=True,\n",
       "         regressors=[SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1,\n",
       "  gamma='auto_deprecated', kernel='linear', max_iter=-1, shrinking=True,\n",
       "  tol=0.001, verbose=False), LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,\n",
       "         normalize=False), Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,\n",
       "   normalize=False, random_state=1, solver='auto', tol=0.001)],\n",
       "         store_train_meta_features=False, use_features_in_secondary=False,\n",
       "         verbose=0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stregr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Stacked Regression and GridSearch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this second example we demonstrate how `StackingCVRegressor` works in combination with `GridSearchCV`. The stack still allows tuning hyper parameters of the base and meta models!\n",
    "\n",
    "For instance, we can use `estimator.get_params().keys()` to get a full list of tunable parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.082505 using {'lasso__alpha': 0.1, 'meta_regressor__C': 1.0, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import Lasso\n",
    "\n",
    "# Initializing models\n",
    "\n",
    "lr = LinearRegression()\n",
    "svr_lin = SVR(kernel='linear')\n",
    "ridge = Ridge(random_state=1)\n",
    "lasso = Lasso(random_state=1)\n",
    "svr_rbf = SVR(kernel='rbf')\n",
    "regressors = [svr_lin, lr, ridge, lasso]\n",
    "stregr = StackingRegressor(regressors=regressors, \n",
    "                           meta_regressor=svr_rbf)\n",
    "\n",
    "params = {'lasso__alpha': [0.1, 1.0, 10.0],\n",
    "          'ridge__alpha': [0.1, 1.0, 10.0],\n",
    "          'svr__C': [0.1, 1.0, 10.0],\n",
    "          'meta_regressor__C': [0.1, 1.0, 10.0, 100.0],\n",
    "          'meta_regressor__gamma': [0.1, 1.0, 10.0]}\n",
    "\n",
    "grid = GridSearchCV(estimator=stregr, \n",
    "                    param_grid=params, \n",
    "                    cv=5,\n",
    "                    refit=True)\n",
    "grid.fit(X, y)\n",
    "\n",
    "print(\"Best: %f using %s\" % (grid.best_score_, grid.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-9.810 +/- 6.86 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 0.1, 'svr__C': 0.1}\n",
      "-9.591 +/- 6.67 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 0.1, 'svr__C': 1.0}\n",
      "-9.591 +/- 6.67 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n",
      "-9.819 +/- 6.87 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 1.0, 'svr__C': 0.1}\n",
      "-9.600 +/- 6.68 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 1.0, 'svr__C': 1.0}\n",
      "-9.600 +/- 6.68 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 1.0, 'svr__C': 10.0}\n",
      "-9.878 +/- 6.91 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 10.0, 'svr__C': 0.1}\n",
      "-9.665 +/- 6.71 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 10.0, 'svr__C': 1.0}\n",
      "-9.665 +/- 6.71 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 10.0, 'svr__C': 10.0}\n",
      "-4.839 +/- 3.98 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 0.1}\n",
      "-3.986 +/- 3.16 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 1.0}\n",
      "-3.986 +/- 3.16 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n",
      "...\n",
      "Best parameters: {'lasso__alpha': 0.1, 'meta_regressor__C': 1.0, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n",
      "Accuracy: -0.08\n"
     ]
    }
   ],
   "source": [
    "cv_keys = ('mean_test_score', 'std_test_score', 'params')\n",
    "\n",
    "for r, _ in enumerate(grid.cv_results_['mean_test_score']):\n",
    "    print(\"%0.3f +/- %0.2f %r\"\n",
    "          % (grid.cv_results_[cv_keys[0]][r],\n",
    "             grid.cv_results_[cv_keys[1]][r] / 2.0,\n",
    "             grid.cv_results_[cv_keys[2]][r]))\n",
    "    if r > 10:\n",
    "        break\n",
    "print('...')\n",
    "\n",
    "print('Best parameters: %s' % grid.best_params_)\n",
    "print('Accuracy: %.2f' % grid.best_score_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error: 0.1844\n",
      "Variance Score: 0.7331\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD1CAYAAAC1BoUqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3XtclGX+//HXAIMgiijJYGgmHvCEmqihbqIouuYBT2SmrrqmuZWua5aaechjbWlruduaZh62rNVUCnLTPGGaB/CA6ISpWYIwKjgiIjLA/P7wx3xFGEBmnGHu+Twfj30sM3Pfc30uoffcc93Xfd0qo9FoRAghhCK52LsAIYQQj46EvBBCKJiEvBBCKJiEvBBCKJiEvBBCKJiEvBBCKJibvQu4X0JCgr1LEEIIhxQSElLq81Uq5MF8oRWh1Wpp0aKFFatxDM7ab5C+O2PfnbXfYL7vZR0gy3CNEEIomIS8EEIomIS8EEIomIS8EEIomIS8EEIoWJWbXSOEJfR6PTqdDoPBgFqtRqPR4OPjY++yhLAbCXmhGHq9ntTUVIpWzzYYDKSmpgJI0AunJcM1QjF0Oh0P3h7BaDSi0+nsVJEQ9mdxyKelpTF69GieffZZ+vXrx/r160tsYzQaWbRoEREREQwYMIAzZ85Y2qwQJRgMhod6XghnYPFwjaurKzNnzqRVq1ZkZ2czdOhQunbtSpMmTUzbxMXFcenSJXbu3MmpU6eYP38+mzdvtrRpIYpRq9WlBrparbZDNUJUDRYfyfv5+dGqVSsAatSoQWBgYImvx7t372bQoEGoVCratWtHVlYWV69etbRpIYrRaDSoVKpiz6lUKjQajZ0qEsL+rHriNSUlBa1WS9u2bYs9r9Pp8Pf3Nz329/dHp9Ph5+dX4j20Wm2l28/NzbVof0flrP2GivU9LS2NtLQ0G1VkO876e3fWfkPl+m61kL99+zZTpkzhzTffpEaNGpV+H0sWHnLWhYuctd8gfXfGvjtrv8GOC5QZDAamTJnCgAED6N27d4nXNRoN6enppsfp6enyFVoIIWzA4pA3Go3Mnj2bwMBAxo0bV+o24eHhbN++HaPRyMmTJ6lZs2apQzVCCCGsy+LhmoSEBKKjo2nWrBmRkZEATJs2jStXrgAwYsQIwsLC2L9/PxEREXh6erJkyRJLmxVCCFEBFod8hw4dSE5OLnMblUrFvHnzLG1KCCHEQ5IrXoUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsEk5IUQQsGsEvKzZs2ic+fO9O/fv9TXjxw5QkhICJGRkURGRrJy5UprNCuEEKIcFt/+D2DIkCGMGjWKGTNmmN2mQ4cOrFq1yhrNCSGEqCCrHMl37NiRWrVqWeOthBBCWJFVjuQr4uTJkwwcOBA/Pz9mzJhB06ZNbdW0Yuj1enQ6HQaDAbVajUajwcfHx95lCSGqMJXRaDRa441SUlKYNGkSMTExJV7Lzs5GpVLh5eXF/v37Wbx4MTt37iyxXUJCAtWrV690Dbm5uXh4eFR6/6qssLCQ0n5VKpWKvLw8xfa7PEr+nZfHWfvurP0G833PyckhJCSk1H1sciRfo0YN089hYWG8/fbbZGZmUqdOnRLbtmjRotLtaLVai/avypKTkzEYDCWed3Nzw8XFRbH9Lo+Sf+flcda+O2u/wXzfExISzO5jkymU165dMx2FJiYmUlhYSO3atW3RtGKUFvBlPS+EEGClI/lp06Zx9OhRbty4Qbdu3Zg8eTL5+fkAjBgxgu+//55Nmzbh6uqKh4cHy5cvR6VSWaNpp6FWq0sNdLVaTWFhoR0qEkI4AquE/PLly8t8fdSoUYwaNcoaTTktjUZDampqsXF5lUqFRqMhLS3NjpUJIaoyueLVQfj4+BAQEIBarQbuHcEHBATI7BohRJlsNoVSWM7Hx0dCXQjxUORIXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFMwqIT9r1iw6d+5M//79S33daDSyaNEiIiIiGDBgAGfOnLFGs0IIIcphlZuGDBkyhFGjRjFjxoxSX4+Li+PSpUvs3LmTU6dOMX/+fDZv3myNpoWd6fV6dDodBoMBFxcXVCoVBQUFqNVqNBqN3ORECDuzSsh37NiRlJQUs6/v3r2bQYMGoVKpaNeuHVlZWVy9ehU/Pz9rNC/sRK/XF7vv7P03FDcYDKSmpgKUCPr7Pxjkw0CIR8smY/I6nQ5/f3/TY39/f3Q6nS2aFo+QTqcrdmPxBxmNxhK/56IPBoPBAPzfh4Fer3+ktQrhrKrcPV61Wm2l983NzbVof0dlr34XFBSUu43BYChWW2n7GI1GUlJSSEtLe+ganPV3Ds7bd2ftN1Su7zYJeY1GQ3p6uulxeno6Go2m1G1btGhR6Xa0Wq1F+zsqe/U7OTnZdERujlqtJigoyPQ4KSnJ7LYV7cODwz316tVzyuEe+Xt3Pub6npCQYHYfmwzXhIeHs337doxGIydPnqRmzZoyHq8AGo0GlUpl9nWVSlXiw1ytVpe6rbnnH/TgcA8gwz1ClMEqR/LTpk3j6NGj3Lhxg27dujF58mTy8/MBGDFiBGFhYezfv5+IiAg8PT1ZsmSJNZoVdlZ09Pwws2s0Gk2xk7VQ+oeBOaWdByga+3fGo3khymOVkF++fHmZr6tUKubNm2eNpkQVkX4znf3n9rMveR/HLh3jVu4tcg255BpyyS/Mx6uaF7U8a+Ht4Y23p7fp51qetXDHHfLB18OXto+3pX2z9hUOaHPDQwaDAb1ez5UrV4rN8nF1dXXa4RwhoAqeeBUPr7Cw0DQ+/qimJF7Nusr+c/vZm7yXfcn70KaVffIn83Yml7lcofeu41WHDg07ENIwhA5PdqBDww40qNOg1KEgtVpdatC7uLiUOo23oKDA7FROIZyBhLwDKzpyNRqNJaYkgmWhdu3WNdOR+t6f93I27Wyx16u7V+cPTf5A96DuPNP0GR6r8Rgeag881B64uriSk5fDzTs3ybqTZfr/rNysYs9duHaBY5eOcT37OjvP7mTn2Z2m969bs26J4H/c53Gzwz1lnRuQ4RzhzCTkHdSDFyLdrzKhdv3WdVOo7zu3j6TU4rNgPN096dq4Kz2CetA9qDsdnuyAu5u7xf0wGo1czrxM/G/xxF+KJ+G3BOJ/i+farWvsSNrBjqQdpm39a/nToWEHWmta07BGQ1rVbUVtz9oEBASUeTEemB/mEULpJOQdVHkXIpUXahnZGcSdizMNv5xOPV3sdQ+1B12bdKV7s+70aN6Djk92tEqoP0ilUvGE7xM84fsEQ9oPAe4F/6Xrl/4v+H9PIP5SPOk304lJjCGGGABcVC60f7w9o/4wija12lDXs67Zdio6e0cIpZGQd1AVmZ9+v8zbmcSdi7t3pJ68j8TUxGIfEtXcqtGlcRd6NO9B92bd6dSoE9XU1R5J7eVRqVQ0qtuIRnUbEdUhCrgX/BeuXTAd7R+7dIxDFw4RnxpP/FfxALTya0WvJr3o2aQnjWo3KvZ+FZ29I4TSSMhXUeWt72LuBCTcC7VqNavxzclvTEfqp1JOlQj1zo07m47UOzXqhIfa45H3q7JUKhVN/JrQxK8Jz3d6HoCbOTdZ/f1qjuiO8N3p7zhz9Qxnrp5hxaEVtKjbgufbPM+g1oN4IuAJGY8XTktCvgp6cLy9tJOpD56AvHX3FgmpCcRfiedE+glOXzldLNTd3dwJDQw1jamHBoZW6VCviFrVa9GveT+mD57Onbw77Dy7k63HtxJ9MhrtNS3zds9j4+mNzB8wn+c7PY+ri6u9SxbC5iTkq6CKXPBT9P9Hfz7K2qNr2a7dzt38u6bt1a5qQgND6R7UnR5BPQgNDMXT3dN2nbAxT3dPIttFEtkuklxDLpvjN7MgZgHnr55n1KejWPLdEt6OfJshTw3BxUXulSOch4R8FVTWBT9FtGlaFscuZtPRTRQa713806VxF8Kbh9M9qDudAztTvVp1m9Rb1XioPRjdeTQjOo1g4+GNvP3t25xNO0vUv6No16AdCyIX0L9N/zKnXQqhFBLydmRu3N3ceLtareZM6hkWxi7kv/H/xWg04ubqxsDmA1n83GJaPt7SDr2outxc3RjXdRwjnx7Jpz9+yqLYRZy8fJKBKwfSqVEnFkYuJKJlhIS9UDT53monZa2rXtrCX+nZ6Szcv5A2b7fhq2Nf4ebixqSwSfyy6BeW9FkiAV8Gdzd3/tL9L5xffJ4Phn+AX00/jv56lD7/6EPYe2HEnYuzd4lCPDIS8nag1+tJSUkpc9w9ICAAtVrNzdyb/OPQP+i/vj+fH/scFxcXXu7+MheXXOTjUR/z5GNP2qcTDsjT3ZOpvaZycelF3hnyDrWr1+bALwcIey+M3h/05sjFI/YuUQirk5C3saIjeHOKjuyr16jOt5e+pf/G/nx67FPu5t9leMfhaBdo+efIf1K/Tn1blaw4XtW8mNF3Br8u/ZX5A+bj7enNrrO7CF0ayoCPBnDi9xP2LlEIq5GQt7HyrlRVq9Xs0e6h3YJ2vL7ldfQ5esKbh3Ns9jG+nPglTfya2LBaZatVvRbzBs7j16W/MqvvLLyqeRGTGEP7he0Z9vEwzqSesXeJQlhMQt7GyrpS9ertq8zaNYuey3uiTdPSxK8J3035jh+m/UCHJzvYsErnUserDkuGLOHikotMi5iGh9qDr49/TfDbwYxcPZJfdL/Yu0QhKk1C3sZKW0PFUGBgXcI6IjdGsu3UNjzdPVk8aDFJ85PoG9xXZn/YiJ+3H8ueW8aFJRd4pccruLm48cXRL2gxtwXj143n0vVL9i5RiIcmIW9jD86cOXr5KFFfRLHsx2Vk381m8FOD0S7Q8ma/N+22doyze9zncVa+sJJfFv/C+D+MB2DtwbU0e6sZL3/+Mqk3zJ9TEaKqkZC3saKZM7fzbzNn1xzGbx3PhcwLpqGZrS9vpaFvQ3uXKYCGvg1ZM2YNPy/8mVGho8gvzOfjfR/T+M3G/O2rv6HL0tm7RCHKZZWQj4uLo0+fPkRERPDJJ5+UeH3r1q2EhoYSGRlJZGQkmzdvtkazDuvH334kcmMk289up5pbNRZELuD0/NP0De5r79JEKZr4NWHj+I0kzU8iKiSKu/l3+ccP/yBwViCzts4iIzvD3iUKYZbFIV9QUMCCBQtYs2YNsbGxxMTEcP78+RLbPfvss0RHRxMdHU1UVJSlzTqkW7m3mLBhAgNWDkCXpeOZps9wat4p5vSf4/CLhTmDlo+35L+T/suJOScY0HYAOXk5vLPjHRrNasT8b+ZzM+emvUsUogSLQz4xMZGGDRvSoEED3N3d6devH7t377ZGbQ5Br9eTnJxMUlISycnJ6PX6Urc7eP4g7Ra0Y82BNVRzq8ayqGXsm76PIP8gG1csLNXuiXZ88+o3HHnzCL1b9uZW7i3e/vZtGs1qxDs73iE7N9veJQphojKWNWm7Av73v/9x4MABFi9eDMD27dtJTExk7ty5pm22bt3K8uXLqV27No0aNWLWrFnUq1evxHslJCRQvXrlF9XKzc3Fw8N2R8SFhYWlznlXqVSmlQ7zCvJYeWgla+PXUmgspHnd5rzb912aPtbUanXYut9VSVXoe3xKPB8e/JD41Hs3L6njWYcJnSYwvM1w3F3dS9yP1lqrYFaFvtuDs/YbzPc9JyeHkJCQUvexyQJlPXr0oH///ri7u/Pll18yY8YMNmzYUOq2LVq0qHQ7Wq3Wov0fVnJycqnz3t3c3AgKCiIpNYkxa8ZwKuUULioXZvWdxfyB861+Gz1b97sqqQp9b9GiBaN6jWK3djdvbX+LI78e4d3977LhxAbGh4xncMvBpt95UeDXq1fP4huZVIW+24Oz9hvM9z0hIcHsPhYfUmg0GtLT002PdTpdiVut1a5dG3f3e3/kUVFRnDnjmFcSPjg0Y+7Cprt5d1m2cxkhi0I4lXKKwLqBxL0Rx5IhSx7JfVKF/alUKnq17MVPs34iZnIMTz3xFGlZaSzau4gBGwYQ83OMKeALCgpMi9EJ8ahZHPLBwcFcunSJy5cvk5eXR2xsLOHh4cW2uXr1qunnPXv20LhxY0ubtbnSVo0szZWsK7y49UWmb55OXn4eE56ZwKm5p+japKstyxV2olKp6NemH/Gz4/mg3wc08W3ClVtXmPX9LF6Ofpn0W/cOiIoWoxPiUbN4uMbNzY25c+fy4osvUlBQwNChQ2natCkrVqygdevW9OzZk40bN7Jnzx5cXV2pVasWS5cutUbtNlXemjNGo5FobTTv7H+H23m30XhrWPOnNfRv29+GVYqqwsXFhb4t+tIjsAffaL/hvQPv8eNvPzL4P4N5o9sbDGo5qNybsQthDVYZkw8LCyMsLKzYc3/9619NP7/22mu89tpr1mjKbsr6D/KW4RZzvp/D7gv3ZhUNfmowq0avom7NurYqT1RBRffhHdxqMM88+QwL9ixg78W9zP1hLrvO72LRHxfZu0ThBOTOUBVk7m5Nh1MO89aut9Bl6ajpUZOPRnzEnzr/SdabEaYTq1euXOExr8dY0X8FscmxLNm3hAOXDjBw/UDe1b9LWEAY+fn5xe4OJoS1yLIGFfTgmjOGAgPvH3ifCV9PQJelI6xZGKfnn2ZMlzES8MLEx8eHli1bUr9+fdzd3enfvD8x42KIaB7BzTs3mfTlJF7e9jLXb18vdncwIaxFjuQrqOjoSqfTcfHaRd743xsk6ZJwdXFl0aBFvN7ndVxdXO1cpaiqfHx8ih2hf9/+e5Z+vZSle5ey9+JeTqWdYmmfpXRp2MV0dzAhrEGO5B+Cj48PJ26eYPhXw0nSJdHQtyEH3jjAzL4zJeDFQ1GpVAxsPpBto7bxdP2nybyTyaTtk/jw0IfcuXvH3uUJBZGQr6CcuzlM2DCBEatHcCv3FkPbD+Xk3JN0btzZ3qUJB6VWq/Gv6c+qwat4JfQVVCoVq4+tZsK2CaRkpti7PKEQEvIVcPbKWTot6WRad+bjkR+zedJmfKrLV2pReUXneVxdXJn09CTWDFlDXa+6xKfE025hO747/Z29SxQKICFfBqPRyNof19JhcQfOXDlDkH8QR948wqTuk+TkqrBY0b0Fiu4W1qVRFw5OP0ifVn3IyM6g34f9eGPLGxjyZT69qDwJeTNu5d5i9KejGb9+PHfy7jCm8xjiZ8fTtkFbe5cmFMTHx4egoCBat25NUFAQTes35bsp37F0yFJcXVx57/v36PZeN37L+M3epQoH5ZQhX97ywCd+P0HIwhA+P/I5XtW8WD9uPev+vI4aHjXsVLFwJi4uLszsO5P9r++nfu36HL54mKcWPEX0yWh7lyYckNOFfGlr0BTNTTYajazcs5LQpaH8cvUX2tRvQ/zseP7U5U92rlo4o65NunJy7kn6t+nPjZwbDPrnIKZ+OZW8/Dx7lyYciNOFfGlr0BiNRs79do6hHw9l8qbJ5OXn8Zfuf+HwrMM0r9fcTpUKAb41fPnm1W9YFrUMN1c3VuxeQdd3unLx2kV7lyYchFNcDKXX69HpdGbXnzmVdoo3drzBlVtX8Pb0Zs2f1hDVwTlvUWip+/+t5TJ961CpVEzrPY0/NP0Dw1cNJ/63eJ5a+BRv93zbaddVFxWn+CP5B4dn7ldoLGRt/FrGbhnLlVtX6PhkR07MOSEBX0llDYUJy3Vq1IkTc08w+KnBZN3J4m8xf+OVz18h15Br79JEFabokNfr9aSkpJS6RHBmTiYvR7/MBwc/IL8wn1e7vcqPM34ksG6gHSpVBnNDYbJuuvX4VPfh6798zUcjPkLtquZf+/5F56Wd+UX3i71LE1WUYkO+6KiyNId/P8ywL4Zx8LeD+Hj68OWfv+Sj0R/JXZssZG44TNZNty6VSsWr4a/yxfNf0LhuY05ePkn7he3ZdGSTvUsTVZBiQ760o0pDgYEPfvyAidsmcu32NZ5p+gyn559meOfhdqpSWYou6qno88IyrTStOD7nOMM7Dif7bjYvrHmBCRsmkHM3x96liSpEsSH/4NHjZf1lxmwew9qEtahUKmb1nsXe6XupX6e+nSpUngeXY4Z7R50P3vNXWI+3pzebJmxi1ehVVHOrxpoDa3h66dNo07T2Lk1UEVYJ+bi4OPr06UNERASffPJJidfz8vKYOnUqERERREVFkZLy6BdfKjp6NBqNbD+7nWFfDOO07jT1atYj5i8xLIlaIitHWtmDl+mr1WoCAgJkds0jplKpmNhtIkdnHyXIP4ik1CQ6LOrA+kPr7V2aqAIsDvmCggIWLFjAmjVriI2NJSYmhvPnzxfbZvPmzXh7e7Nr1y7Gjh3L+++/b2mz5dJoNGTdzWL6junM2TWHHEMOfZr24dD0Q/R9qu8jb99ZPXiZvgS87RRdvDc6dDQ5eTmM/WwsY9aO4fbd2/YuTdiRxSGfmJhIw4YNadCgAe7u7vTr14/du3cX22bPnj0MHjwYgD59+vDTTz+VeVNsa/jp8k8M/WIoO3/ZSXV1dZb+cSmbJmziyceffKTtCmFPNTxqsP7P61k7di2e7p5s+GkDHRZ14HTKaXuXJuzE4pDX6XT4+/ubHms0mhJT5nQ6HfXq1QPAzc2NmjVrcuPGDUubLlV2bjYvbXyJZz98lvSsdLo07kLi/ERmDp1J7dq1H0mbQlQlKpWKcV3HET87npb1WvJz+s+mpbIf9cGVqHqq3BWvWm3lTxj99OtP9Pm0D5dvXkbtqmZKlymMDRlLXkYe2gzlnojKzc216N/NkUnfzfddhYqNwzayeM9itp7ZyoQNE9h+ZDvzI+bj5e5VYvvCwsJiHwIqlQoXl6o3N0N+5w/Xd4tDXqPRkJ6ebnqs0+lKzKbQaDSkpaXh7+9Pfn4+t27dMntUXdnLtOdFz2NhzEKMGGlbvy0bx28kuH5wpd7L0Wi1Wqe9vF36Xn7fv27zNf85/B8m/WcSscmxJN9I5r8v/ZennnjKtI2560rq1atX5c6ryO+8ZN8TEhLM7mPxx3RwcDCXLl3i8uXL5OXlERsbS3h4eLFtwsPD2bZtGwDff/89oaGhVr3pRtadLBbELEClUvHms29ydPZRpwl4ISpiVOgoEt5KoE39Npy/ep7QpaH8a++/TEfucrWyclkc8m5ubsydO5cXX3yRZ599lr59+9K0aVNWrFhhOgE7bNgw9Ho9ERERfPbZZ0yfPt3iwu/n7enND9N+4OtRX7N48GK5clWIUgT5B3F41mEmhU0iLz+PV754hedWPcfNnJtytbKCWWVMPiwsjLCwsGLP/fWvfzX9XK1aNT788ENrNGVWzxY90eKc43RCVJSnuycfj/qYHkE9eHHDi2xJ2ELCbwn8vc/faf5YyWW15Wplx1f1zqoIIR655zo+x4k5JwhpGMKv13/lhS9f4D8n/lPixKtcrez4JOSFcFKN/RpzcMZBJodPxlBg4N24d5kaO5WbuTflamUFkZAXwolVU1fjwxEfsvUvW/Gp7sOeC3t4YfML6NV6CXiFkJAXQjC4/WBOzDlBp0ad+D3zd575+zOs+GGFXDylABLyQggAnnzsSQ68cYCpvaZiKDAw9aupDPv3MG7m3LR3acICEvJCCBN3N3c+GP4BWyZtwdvTm63Ht9J+UXsSfjN/sY2o2iTkhRAlDA0ZyvG3jvPUE09x8dpFurzThZV7VsrwjQOSkK8kvV5PcnIySUlJJCcny82qheL4uvvyaeSnDA8eTl5+HpM3TSbyw0gyb2fauzTxECTkK6FonY+iqwENBgOpqakS9EIx9Ho9KSkpqF3UvBX+Fu/1fY8a7jX4NulbWr3Vim2Httm7RFFBEvKVIOt8CKV78G/5j83+yOYXNtPGvw3p2ekMWzeM6Z9Pp6CwwE4VPhpK/IYuIV8Jss6HULrS/pbr16rPumHrGN9hPEajkWX7ltFzWU9SMh/97TxtQanf0CXkK8Hceh6yzodQCrN/465qpnadyqrBq3is+mPsP7eftgvaEn0y2sYVWp9Sv6FLyFeCRqMpsVSyrPMhlKS8v+XOT3Rmy8gt9G3dl8zbmQz65yAmfzGZXEOujSq0PqV+Q5eQrwQfHx8CAgJMRzuyzodQGh8fH+rXr1/mnaGaBDQhZnIMy59bjtpVzcq9K3l6ydMkpSbZsFLrUeo3dAn5SvLx8SEoKIjWrVsTFBQkAS8Ux8fHh5YtW9K6desSd3KrXbs2AQEBuLi48LeIv/HTrJ9o4teExJREQhaF8Pf//d3hTsoq9Ru6hLwQolwBAQG0bt3a9L+AgIBir4c0DOH4nONM7DaRvPw8Znw9g7D3wjh/9bydKn54Sv2GLiEvhLCKmh41WTV6Fd9N+Y56tepx8PxB2r7dtthtBqs6JX5Dtyjk9Xo948aNo3fv3owbN46bN0tfyKhFixZERkYSGRnJpEmTLGlSCFHF9Q3uS9LbSbzQ6QVy8nJ45YtX6POPPlzOvGzv0pySRSH/ySef0LlzZ3bu3Ennzp355JNPSt3Ow8OD6OhooqOj+fe//21Jk0IIB1DHqw6fT/iczZM241vDl11ndxE8P5gNhzY4zFG9UlgU8rt372bQoEEADBo0iB9++MEqRQkhlGFYyDCS5icxoO0Abt65yZjPxjDkX0PQZTn23HNHYlHIZ2Rk4OfnB0DdunXJyMgodbu7d+8yZMgQnnvuOfkgEMLJ+NfyJ/qVaD4b+xnent5sP7mdoLeCWPHDCgz5jj0H3RGojOV8dxo7dizXr18v8fzUqVOZOXMm8fHxpuc6duzIsWPHSmyr0+nQaDRcvnyZMWPGsG7dOp544okS2yUkJFC9evXK9AOA3NxcPDw8Kr2/o3LWfoP03dH6fiXrCvN/mM+Pl34EoIlvE97s8SahT4RW+D0csd/WYq7vOTk5hISElLqPW3lvum7dOrOv+fr6cvXqVfz8/Lh69Sp16tQpdbuieaYNGjSgU6dOnD17ttSQh3sedtD5AAAOCklEQVQnaStLq9VatL+jctZ+g/Td0fpeT1+P1bVWs+vcLt6Le4/zGef585Y/MyxkGMuilvGEb+m5cD9H7Le1mOt7QoL5m7pYNFwTHh7O9u3bAdi+fTs9e/Yssc3NmzfJy8sDIDMzk+PHj9OkSRNLmhVCOKCiBcDy8/PpEdiDbaO2MbnLZKqrq7MlYQvN5zZnwbcLuJN3x96lKopFIT9x4kQOHjxI7969OXToEBMnTgTg9OnTzJ49G4ALFy4wdOhQBg4cyJgxY5gwYYKEvBBO6MEFwKq5VWNix4nEjotleMfh3Mm7w7xv5tFybku2n9gus3CspNzhmrLUrl2b9evXl3g+ODiY4OBgANq3b8+3335rSTNCCAUwt9DXY56P8eXEL5kUNokpm6ZwOvU0g/81mN4te7Pi+RU0r9fcxpUqi1zxKoSwifIWAOse1J3jc47z0YiP8Knuw86zOwl+O5jpm6eTdSfLlqUqioS8EMImKrIAmJurG6+Gv8q5ReeY2G0iBYUFLNu5jGZvNWP9ofUUFhbaumyHJyEvhLCJh1kArG7NuqwavYr42fF0adwFXZaOsZ+Npeu7XUlKd8yljO1FQl4IYTMPuwBY+4bt+XHGj2z48wb8a/lz+OJhhn8xnOc/eZ7jvx23UdWOTUJeCFGlqVQqRncezblF53i9z+u4urjy1bGvCFkUQsTyCHad3SUzccogIS+EcAg1PWry92F/Z+f4nbzW+zVqVKvBD9of6P1Bb0IWhfDl0S/JL8i3d5lVjoS8EMJh6PV66lavy9gWY9k9YTdz+s5B463hxO8nGLF6BM3easY/9/6TnLs59i61ypCQF0I4hKIrZotUd63O8KDhnJx5klWjV9HUrym/Xv+VV794lYYzG7Lg2wVkZJe+aKIzkZAXQjiEB6+YBTAajdzMvMnEbhPRLtSyZdIWOj7ZkevZ15n3zTwCXg9g0D8H8fnhz512rr2EvBDCIZi7YrboeVcXV4aGDOXIm0fYN30ffVv3Ja8gj+iT0Yz6dBR1p9Vl4MqBbPxpIzdzSr+LnRJZtKyBEELYilqtLjXoH7ySVqVSERYURlhQGFf0V9h2Yhub4zcT90sc3576lm9PfYvaVU3vlr2J6hDFwLYDqe1V21bdsDkJeSGEQ9BoNKSmphYbsnnwitkier0enU6HwWCg1+O9GDlhJLmqXLYe38qWhC3sP7ef2NOxxJ6ORe2qpleLXkR1iCKyXSR1vEpfMt1RScgLIRxC0YVTKSkpwL0jeI1GU+KCqqITtEUfBgaDgdTUVAICAni5x8u83ONldFk6th3fxpbjW9j78152JO1gR9IO3Fzd6BHUgx5BPQgNDKXDkx2o6VHTth21Mgl5IYTD8PHxIS0trcybhpg7QavT6UwfCBpvDZO6T2JS90lcu3XNNKSzN3kvu87uYtfZXQC4qFxo9XgrQgNDafV4K3xr+OLr5YtvDV/qeNXB18uXWp61cHGpuqc3JeSFEIpS3gnaB9WtWZeJ3SYysdtErt+6zvdnvueniz9x5OIRTqac5HTqaU6nnjbbnovKhTpede6FftGHgJdv8cf3fSgU/Ww0GsnKzeJW7i3q1aqHt6e3Vfr/IAl5IYSiVPQEbWkeq/kYI0NHMjJ0JAB38u5w/PfjHLl4hF+v/0rG7QwysjPIvJ157+fbGWTdyeJ69nWuZ18HXeVqfqzGY6S+l4q7m3vl3qAMEvJCCEV5mBO099Pr9Vy5csW0nLGrqyv16tWja5OudG3S1ex+hnwDmTmZ94I/O6PkB8H/f+7B111dXPH29KZmtZr8oekfULuW/yFUGRLyQghFKRp3L5pdY+4E7f30er3phG6RgoICUlNTuX37NtnZ2WbfS+2mRuOtQeNd9oeIvVgU8jt27GDlypVcuHCBzZs3m27596C4uDgWL15MYWEhUVFRpnvBCiHEo+Dj41PuMsb30+lKH2cxGo3cuHHD9Lhopk5RG5Vx//ROFxcXVCoVBQUFFfowqgyLTgk3a9aMjz76iI4dO5rdpqCggAULFrBmzRpiY2OJiYnh/PnzljQrhBBWZe6kbGmKZupURtH0zqL2CgsLKSgoMNWQmpqKXq+v1HubY1HIN27cmMDAwDK3SUxMpGHDhjRo0AB3d3f69evH7t27LWlWCCGsqiInZe/3MB8K9ytteuf9LPkAMeeRj8nrdDr8/f1NjzUaDYmJiWa312q1lW4rNzfXov0dlbP2G6Tvztj3R9Hvytw7tjI1FB21l8VgMJh978r0vdyQHzt2LNevXy/x/NSpU+nVq9dDNVYRZV3kUB6tVmvR/o7KWfsN0ndn7Puj6ndps2u8vb3R6/UlZuqYuzdteZKTk8v9FqBWqwkKCir1NXN9T0hIMPt+5Yb8unXrytukTBqNhvT0dNNjnU5X7lQmIYSwNXMna728vB5qpk5ZSpveeb+KTPV8WI98uCY4OJhLly5x+fJlNBoNsbGxLFu27FE3K4QQVvGwM3XKey/AprNrLAr5Xbt2sXDhQjIzM3nppZdo0aIFn376KTqdjrfeeovVq1fj5ubG3LlzefHFFykoKGDo0KE0bdrUWvULIYRDseaHRkVYFPIRERFERESUeF6j0bB69WrT47CwMMLCwixpSgghRCVU3aXThBBCWExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFExCXgghFEwRIa/X60lOTqagoIDk5GT0er29SxJCiCrBojtD7dixg5UrV3LhwgU2b95McHBwqduFh4fj5eWFi4sLrq6ubN261ZJmi9Hr9cVujGswGEhNTQWw6S22hBCiKrIo5Js1a8ZHH33EvHnzyt12/fr11KlTx5LmSqXT6Urc+dxoNKLT6STkhRBOz6KQb9y4sbXqqDSDwfBQzwshhDOx2Zj8+PHjGTJkCF999ZVV31etVj/U80II4UzKPZIfO3Ys169fL/H81KlT6dWrV4Ua2bRpExqNhoyMDMaNG0dgYCAdO3YsdVutVluh9yxSWFhY6vP5+fkP/V6OKjc312n6+iDpu/P13Vn7DZXre7khv27dusrWY6LRaADw9fUlIiKCxMREsyHfokWLh35/vV6PTqfDYDCgVqvRaDRONR6v1Wor9e+mBNJ35+u7s/YbzPc9ISHB7D6PfLgmJyeH7Oxs088HDx6kadOmVm3Dx8eHoKAgXF1dCQoKcqqAF0KIslgU8rt27aJbt26cOHGCl156ifHjxwP3ZrxMmDABgIyMDF544QUGDhxIVFQUYWFhdOvWzfLKhRBClMui2TURERFERESUeF6j0bB69WoAGjRowDfffGNJM0IIISpJEVe8CiGEKJ1FR/JCCAEy+aEqk5AXQlhElhap2mS4RghhkbKWFhH2JyEvhLCILC1StUnICyEsIkuLVG0S8kIIi2g0GlQqVbHnVCqV6Up3YV9y4lUIYZGik6syu6ZqkpAXQljMx8dHQr2KkuEaIYRQMAl5IYRQMAl5IYRQMAl5IYRQMAl5IYRQMJXxweuR7aisu5sIIYQwLyQkpNTnq1TICyGEsC4ZrhFCCAWTkBdCCAVTxBWvcXFxLF68mMLCQqKiopg4caK9S7KJWbNmsW/fPnx9fYmJibF3OTaVlpbGG2+8QUZGBiqViueee44xY8bYu6xH7u7du4wcOZK8vDwKCgro06cPU6ZMsXdZNlVQUMDQoUPRaDSsWrXK3uXYTHh4OF5eXri4uODq6srWrVsrtqPRweXn5xt79uxp/P3334137941DhgwwPjLL7/YuyybOHr0qDEpKcnYr18/e5diczqdzpiUlGQ0Go3GW7duGXv37u0Uv/fCwkJjdna20Wg0GvPy8ozDhg0znjhxws5V2dbatWuN06ZNM06cONHepdhUjx49jBkZGQ+9n8MP1yQmJtKwYUMaNGiAu7s7/fr1Y/fu3fYuyyY6duxIrVq17F2GXfj5+dGqVSsAatSoQWBgoFPcpEKlUuHl5QVAfn4++fn5JVaAVLL09HT27dvHsGHD7F2Kw3D4kNfpdPj7+5seazQap/iPXfyflJQUtFotbdu2tXcpNlFQUEBkZCRdunShS5cuTtNvgCVLlvD666/j4uLw0VUp48ePZ8iQIXz11VcV3sc5/6WEYty+fZspU6bw5ptvUqNGDXuXYxOurq5ER0ezf/9+EhMTOXfunL1Lsom9e/dSp04dWrdube9S7GLTpk1s27aN1atX8/nnn3Ps2LEK7efwIa/RaEhPTzc91ul0crMCJ2EwGJgyZQoDBgygd+/e9i7H5ry9vXn66ac5cOCAvUuxiePHj7Nnzx7Cw8OZNm0ahw8fZvr06fYuy2aKcs3X15eIiAgSExMrtJ/Dh3xwcDCXLl3i8uXL5OXlERsbS3h4uL3LEo+Y0Whk9uzZBAYGMm7cOHuXYzOZmZlkZWUBkJuby6FDhwgMDLRzVbbx2muvERcXx549e1i+fDmhoaG8//779i7LJnJycsjOzjb9fPDgQZo2bVqhfR1+CqWbmxtz587lxRdfNE2tqmjnHd20adM4evQoN27coFu3bkyePJmoqCh7l2UTCQkJREdH06xZMyIjI4F7/x5hYWF2ruzRunr1KjNnzqSgoACj0cgf//hHevToYe+yxCOWkZHBK6+8Atw7J9O/f3+6detWoX1lWQMhhFAwhx+uEUIIYZ6EvBBCKJiEvBBCKJiEvBBCKJiEvBBCKJiEvBBCKJiEvBBCKJiEvBBCKNj/A23kDcbdMHzLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Evaluate and visualize the fit\n",
    "print(\"Mean Squared Error: %.4f\"\n",
    "      % np.mean((grid.predict(X) - y) ** 2))\n",
    "print('Variance Score: %.4f' % grid.score(X, y))\n",
    "\n",
    "with plt.style.context(('seaborn-whitegrid')):\n",
    "    plt.scatter(X, y, c='lightgray')\n",
    "    plt.plot(X, grid.predict(X), c='darkgreen', lw=2)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**\n",
    "\n",
    "The `StackingCVRegressor` also enables grid search over the `regressors` and even a single base regressor. When there are level-mixed hyperparameters, `GridSearchCV` will try to replace hyperparameters in a top-down order, i.e., `regressors` -> single base regressor -> regressor hyperparameter. For instance, given a hyperparameter grid such as\n",
    "\n",
    "    params = {'randomforestregressor__n_estimators': [1, 100],\n",
    "    'regressors': [(regr1, regr1, regr1), (regr2, regr3)]}\n",
    "    \n",
    "it will first use the instance settings of either `(regr1, regr2, regr3)` or `(regr2, regr3)` . Then it will replace the `'n_estimators'` settings for a matching regressor based on `'randomforestregressor__n_estimators': [1, 100]`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## StackingRegressor\n",
      "\n",
      "*StackingRegressor(regressors, meta_regressor, verbose=0, use_features_in_secondary=False, store_train_meta_features=False, refit=True)*\n",
      "\n",
      "A Stacking regressor for scikit-learn estimators for regression.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `regressors` : array-like, shape = [n_regressors]\n",
      "\n",
      "    A list of regressors.\n",
      "    Invoking the `fit` method on the `StackingRegressor` will fit clones\n",
      "    of those original regressors that will\n",
      "    be stored in the class attribute\n",
      "    `self.regr_`.\n",
      "\n",
      "- `meta_regressor` : object\n",
      "\n",
      "    The meta-regressor to be fitted on the ensemble of\n",
      "    regressors\n",
      "\n",
      "- `verbose` : int, optional (default=0)\n",
      "\n",
      "    Controls the verbosity of the building process.\n",
      "    - `verbose=0` (default): Prints nothing\n",
      "    - `verbose=1`: Prints the number & name of the regressor being fitted\n",
      "    - `verbose=2`: Prints info about the parameters of the\n",
      "    regressor being fitted\n",
      "    - `verbose>2`: Changes `verbose` param of the underlying regressor to\n",
      "    self.verbose - 2\n",
      "\n",
      "- `use_features_in_secondary` : bool (default: False)\n",
      "\n",
      "    If True, the meta-regressor will be trained both on\n",
      "    the predictions of the original regressors and the\n",
      "    original dataset.\n",
      "    If False, the meta-regressor will be trained only on\n",
      "    the predictions of the original regressors.\n",
      "\n",
      "- `store_train_meta_features` : bool (default: False)\n",
      "\n",
      "    If True, the meta-features computed from the training data\n",
      "    used for fitting the\n",
      "    meta-regressor stored in the `self.train_meta_features_` array,\n",
      "    which can be\n",
      "    accessed after calling `fit`.\n",
      "\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `regr_` : list, shape=[n_regressors]\n",
      "\n",
      "    Fitted regressors (clones of the original regressors)\n",
      "\n",
      "- `meta_regr_` : estimator\n",
      "\n",
      "    Fitted meta-regressor (clone of the original meta-estimator)\n",
      "\n",
      "- `coef_` : array-like, shape = [n_features]\n",
      "\n",
      "    Model coefficients of the fitted meta-estimator\n",
      "\n",
      "- `intercept_` : float\n",
      "\n",
      "    Intercept of the fitted meta-estimator\n",
      "\n",
      "- `train_meta_features` : numpy array, shape = [n_samples, len(self.regressors)]\n",
      "\n",
      "    meta-features for training data, where n_samples is the\n",
      "    number of samples\n",
      "    in training data and len(self.regressors) is the number of regressors.\n",
      "\n",
      "- `refit` : bool (default: True)\n",
      "\n",
      "    Clones the regressors for stacking regression if True (default)\n",
      "    or else uses the original ones, which will be refitted on the dataset\n",
      "    upon calling the `fit` method. Setting refit=False is\n",
      "    recommended if you are working with estimators that are supporting\n",
      "    the scikit-learn fit/predict API interface but are not compatible\n",
      "    to scikit-learn's `clone` function.\n",
      "\n",
      "**Examples**\n",
      "\n",
      "For usage examples, please see\n",
      "    [http://rasbt.github.io/mlxtend/user_guide/regressor/StackingRegressor/](http://rasbt.github.io/mlxtend/user_guide/regressor/StackingRegressor/)\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y, sample_weight=None)*\n",
      "\n",
      "Learn weight coefficients from training data for each regressor.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "- `y` : array-like, shape = [n_samples] or [n_samples, n_targets]\n",
      "\n",
      "    Target values.\n",
      "\n",
      "- `sample_weight` : array-like, shape = [n_samples], optional\n",
      "\n",
      "    Sample weights passed as sample_weights to each regressor\n",
      "    in the regressors list as well as the meta_regressor.\n",
      "    Raises error if some regressor does not support\n",
      "    sample_weight in the fit() method.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit_transform(X, y=None, **fit_params)*\n",
      "\n",
      "Fit to data, then transform it.\n",
      "\n",
      "Fits transformer to X and y with optional parameters fit_params\n",
      "and returns a transformed version of X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : numpy array of shape [n_samples, n_features]\n",
      "\n",
      "    Training set.\n",
      "\n",
      "\n",
      "- `y` : numpy array of shape [n_samples]\n",
      "\n",
      "    Target values.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `X_new` : numpy array of shape [n_samples, n_features_new]\n",
      "\n",
      "    Transformed array.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*get_params(deep=True)*\n",
      "\n",
      "Return estimator parameter names for GridSearch support.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict(X)*\n",
      "\n",
      "Predict target values for X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `y_target` : array-like, shape = [n_samples] or [n_samples, n_targets]\n",
      "\n",
      "    Predicted target values.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict_meta_features(X)*\n",
      "\n",
      "Get meta-features of test-data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : numpy array, shape = [n_samples, n_features]\n",
      "\n",
      "    Test vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `meta-features` : numpy array, shape = [n_samples, len(self.regressors)]\n",
      "\n",
      "    meta-features for test data, where n_samples is the number of\n",
      "    samples in test data and len(self.regressors) is the number\n",
      "    of regressors.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*score(X, y, sample_weight=None)*\n",
      "\n",
      "Returns the coefficient of determination R^2 of the prediction.\n",
      "\n",
      "The coefficient R^2 is defined as (1 - u/v), where u is the residual\n",
      "sum of squares ((y_true - y_pred) ** 2).sum() and v is the total\n",
      "sum of squares ((y_true - y_true.mean()) ** 2).sum().\n",
      "\n",
      "The best possible score is 1.0 and it can be negative (because the\n",
      "\n",
      "model can be arbitrarily worse). A constant model that always\n",
      "predicts the expected value of y, disregarding the input features,\n",
      "would get a R^2 score of 0.0.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : array-like, shape = (n_samples, n_features)\n",
      "\n",
      "    Test samples.\n",
      "\n",
      "\n",
      "- `y` : array-like, shape = (n_samples) or (n_samples, n_outputs)\n",
      "\n",
      "    True values for X.\n",
      "\n",
      "\n",
      "- `sample_weight` : array-like, shape = [n_samples], optional\n",
      "\n",
      "    Sample weights.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `score` : float\n",
      "\n",
      "    R^2 of self.predict(X) wrt. y.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*set_params(**params)*\n",
      "\n",
      "Set the parameters of this estimator.\n",
      "\n",
      "The method works on simple estimators as well as on nested objects\n",
      "(such as pipelines). The latter have parameters of the form\n",
      "``<component>__<parameter>`` so that it's possible to update each\n",
      "component of a nested object.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "self\n",
      "\n",
      "### Properties\n",
      "\n",
      "<hr>\n",
      "\n",
      "*coef_*\n",
      "\n",
      "None\n",
      "\n",
      "<hr>\n",
      "\n",
      "*intercept_*\n",
      "\n",
      "None\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.regressor/StackingRegressor.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
